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DATA TRANSMISSION BY WATERMARK PROXY 

Related Application Data 

This application is a continuation-in-part of copending application , 

filed January 26, 2000, entitled Data Transmission by Watermark Proxy, attorney docket 
60099, which is a continuation-in-part of copending application 09/473,396, filed 
December 28, Y999 entitled Watermark-Based Object Linking and Embedding, the 
disclosure of which is attached as Appendix A. This application is also a continuation-in- 
part of copending ap^ication 09/476,686, filed December 30, 1999, entitled Watermark- 
Based Personal Audio Appliance, the disclosure of which is attached as Appendix B. 
This application is also a continuation in part of copending application 60/134,782, filed 
May 19, 1999, the disclosure o^ which is attached as Appendix C. 

Field of the Invention 
The present invention relates to data transmission, and more particularly relates to 
use of watermarks as proxies for data in transmission. 

Summary of the Invention 
As detailed in the assignee's prior applications, including 60/134,782, 60/141,538, 
and 09/343,104, digital watermark technology has numerous applications beyond its 
traditional role of simply communicating copyright information. One futuristic view 
foresees that all "content" should be watermarked, thereby enabling a great variety of 
operations and transactions whenever watermarked content is processed by digital 
devices equipped with watermark recognition and reading technology. All physical 
media objects can thereby be inherently and persistently digitally-enabled, permitting 
greatly simplified access to networks and execution of local and remote applications. 
The continuing growth of the Internet and beginnings of trends toward pervasive 
computing signal an opportunity to radically change the relationships between traditional 
media content and digital processing environments. 
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In this specification, content refers not just to electronic audio, image, and video 
files, but also includes the content aspects of physical objects and media, e.g., artwork, 
patterns, and labels on product packaging, concert tickets, etc. 

In accordance with a preferred embodiment of the present invention, the 
5 processing of watermark data as pointer to shared resources is sometimes used in lieu of 
transmitting from point to point the object with which it is associated, thereby gaining 
efficiencies in speed and bandwidth. 

This and other features and advantages of the present invention will be more 
readily apparent from the following detailed description, which proceeds with reference 
10 to the accompanying drawings. 

Brief Description of the Drawings 
Fig. 1 is a diagram illustrating a network environment in which principles of the 
present invention may be employed. 
1 5 Fig. 2 is a flow chart illustrating aspects of one embodiment of the present 

invention. 

Fig. 3 is a flow chart illustrating aspects of another embodiment of the present 
invention. 

Fig. 4 is a flow chart illustrating aspects of yet another embodiment of the present 
20 invention. 




Detailed Description 
Referring to Fig. 1, consider an exemplary network 10 linking two devices - a 
first device 12 associated with an originating user, and a second device 14 associated 
25 with a recipient user. The first device 12 is coupled to the network through a relatively 
low bandwidth channel, whereas the second device 14 is coupled to the network through 
a relatively high bandwidth channel. (For example, the first device may be an internet- 
capable cell phone having low resolution-, still image only- capture capabilities, 
providing a 9600 baud data channel, or it may be a home PC, with an associated PC or 
30 digital single shot camera, coupled to the internet with a 28.8 kbps modem. The second 
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device may be a computer coupled to the internet through a 1 .45 megabit per second T-l 
line, a cable modem, etc.) The network 10 connecting the two devices includes various 
links - narrow bandwidth at some parts (e.g., 16), very broadband at other (e.g., internet 
backbone 18), etc. 

5 Assume the user of device 12 encounters a printed image, e.g., an advertisement 

in a magazine, that may be of interest to the user of device 12. Using an imaging device 
(e.g., a CMOS- or CCD-camera built into a cell phone, a flatbed scanner connected to a 
PC, etc.), device 12 captures an image of the advertisement. 

In prior art techniques, the image captured by device 12 would have been sent to 
10 device 14 over the network; the image received by the second device would be exactly 
the image sent by the first device. 

In accordance with one embodiment of the invention, device 14 receives a better 
image than that sent from device 12. In one such embodiment, device 14 receives the 
image data captured by device 12. Device 14 recognizes that the image includes a 
15 watermark hidden within the image data, and decodes same. The watermark payload 
includes an index by which a copy of the image can be accessed from a server 20 on the 
internet or other storage medium. With this index, the second device 14 queries the 
server 20, which returns the image corresponding to this watermark index (in this case, 
the advertisement) back to the second device 14. The image provided by the server can 
20 be higher resolution or pristine, i.e., it has no artifacts left from scanning at device 12, etc. 
Such a procedure is shown by the flowchart of Fig. 2. 

watermark payload identifying the sensed image can as long or as short as the 
application requires. Typically, payloads of between 16 and 64 bits are used, although 
this is not essenttd. Shorter payloads have the advantage that they can be more robustly 
encoded while mainfouning a fixed degree of image quality; longer payloads offer a 
greater universe of identifiers with which the image can be labeled. Illustrative 
watermarking technology isSletailed in the assignee's patent 5,862,260, and in copending 

application , fiW February 14, 2000, entitled Watermark Embedder and 

Reader (attorney docket 601 12). vVgreat variety of other watermarking arrangements may 
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be used, includiite those proposed in patents 5,930,369, 5,933,798, 5,664,018, 5,825,892, 
5,940,429 and 5,^9,868. 

In accordance with another embodiment of the invention (Fig. 3), the bandwidth 
bottleneck imposed by narrowband channel 16 (through which device 12 is coupled) is 
5 obviated by employing a watermark as a proxy for an image. In such an arrangement, the 
image data captured by device 12 is decoded, and a watermark payload hidden in the 
image is extracted. (This can be performed by hardware or software available in 
device 12, e.g., a cell phone microprocessor, a desktop computer, dedicated decoder 
circuitry, etc. Alternatively, this decoding can be done remotely from device 12, but 

10 before device 14, e.g., by a smart router in the intervening network. In the following 

discussion, decoding in the device 12 is assumed.) Instead of transmitting the image data 
over the network, the watermark decoding device (e.g., device 12) simply transmits the 
watermark payload (or a part thereof). On receipt of the payload, device 14 again queries 
the server 20, and obtains the image (and/or additional content or functionality, as 

1 5 detailed below) , corresponding to that watermark. The image is obtained over the high- 
speed channel(s) between the server and the second device; the low bandwidth channel 
linking the first device conveys just the low bandwidth watermark payload information. 

By building filters into the low bandwidth devices, upon recognition of a class of 
watermarks indicating availability of the image as a shared resource, or upon user 

20 selection of "transmit only watermark data", the image [or content associated with it via 
the watermark] can be made available to the message recipient via more capable 
transmission means. 

A variant of the foregoing does not transmit the watermark payload to the second 
device 14. Instead, the payload is dispatched by the first device 12 (or the smart router) 
25 directly to the server 20, with instructions that the corresponding desired image be sent to 
the second device 14. Such an arrangement is shown in Fig. 4. 

In some applications, the media delivered by the server may be richer than the 
simple image captured by device 12. For example, the watermark payload in the image 
captured by device 12 may index one or more files on server 20 that includes video, 
30 animation, sound, executable applications, aplets (e.g., JAVA, ActiveX) etc ("enhanced 




b 



WYC:dks 60114 02/15/00 



-5- 



EXPRESS MAIL EL 525675625US 



content"). Thus, scanning of a magazine ad at one device can prompt delivery of a video, 
a Macromedia ShockWave presentation, etc., to the second device. 

In some embodiments, the second device 14 identifies to the server 20 its media- 
playback capabilities. The server 20 can then respond to a watermark-based query with 
5 media appropriate to that particular media consumer. 

One way the media capabilities of device 14 can be indicated to server 20 is by a 
data word comprising flag bits, with each set "1" bit indicating a capability. A simplified 
8 -bit capability word may be as follows: 



Bit 


Capability 


0 


GIF file display 


1 


TIFF file display 


2 


JPEG filed display 


3 


AVI movie display 


4 


WAV sound 


5 


RealAudio sound 


6 


MP3 sound 


7 


WindowsMedia 



" The data comprising this word may be automatically compiled on device 14, e.g., from 
1 0 the operating system database with which programs are registered on installation (the 
Registry database in Windows). 

If device 14 sends the capability word 10101 100 to server 20, the server knows 
the device 14 supports GIF and JPEG imagery (but not TIFF), and RealAudio and WAV 
sound (but not MP3 or WindowsMedia). 
1 5 If server 20 has media content corresponding to the queried watermark in several 

supported formats, it can deliver certain ones according to a priority order (e.g., send 
JPEG if supported; else send GIF if supported; else send TIFF if supported). 

If the server 20 only has media in a format not supported by the second device 14 
(e.g., TIFF in the foregoing example), the server may invoke a conversion routine to 
20 perform an on-the-fly conversion to a supported media type (e.g., JPEG) prior to sending 
to the second device 14. 
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If the watermark index is provided by the second device 14 (rather than directly 
from the first device 12), the capability data word can accompany the index. 

If the watermark index is provided directly from the first device 12, the server can 
solicit from the second device 14 a data capability word before responding to the query. 
5 Alternatively, the server can keep, on-file, a database detailing the media capabilities of 
all known media consumers, and can tailor its query response according to such profile. 
(The second device 14 can be arranged to automatically inform server 20 of updates to its 
capability, e.g., each time a new media playback application is registered in the registry 
database.) 

10 If the server 20 does not know, and cannot discern, the media capabilities of the 

second device 14, it can provide media in a default form that is most likely to be 
acceptable (e.g., JPEG, if the content captured by the first device 12 is imagery). 

From the foregoing description, it will be apparent that embodiments of the 
present invention provide various advantages over the prior art. One is the dispatch of 
1 5 high bandwidth enhanced content using a low bandwidth channel. Another is the receipt 
of higher-quality data than that originally captured. Another is delivering applications 
via low bandwidth channels to recipients by capturing images or watermark data from 
media content that serve as proxies for the applications. 

Having described and illustrated the principles of our invention with reference to 
20 a specific embodiment, it will be recognized that the principles thereof can be 
implemented in other, different, forms. 

For example, while the invention has been described with reference to images, the 
same principles are equally applicable to video and audio. 

Similarly, while the foregoing description has made reference to transmitting the 
25 watermark, in many implementations only a part of the watermark need be transmitted. 
(The watermark may include error correcting information, or other data, not necessary to 
identify the corresponding data on the server 20.) 

Still further, while the detailed embodiment contemplated a still or video camera 
system for first device 12, much of the functionality of such an image capture system 
30 isn't essential to the present invention. Instead, an input device that serves a simpler 
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"watermark capture" function may be used instead. Such a device can omit, e.g., 
hardware or software components associated with pixel interpolation (commonly used to 
achieve a desired virtual resolution), formatting (e.g., to provide output in JPEG form), 
etc. Such components serve useful functions when the resulting imagery is to be 
5 displayed or printed, but are superfluous - or detrimental - when the image data is simply 
to be decoded to extract watermark data. 

While the invention is illustrated with reference to steganographic watermark 
technology for identifying the initial content (i.e., that sensed by device 12), other 
technologies can alternatively be used. These include data glyphs, 1- and 2-D barcodes, 

10 magnetic ink, RF ID tags, UV or IR markings, etc. 

While the detailed embodiment contemplated a single server 20 to serve as the 
repository of content corresponding to watermarks, in other embodiments such a server is 
implemented in distributed fashion. In some embodiments, one server may act as a 
default repository, and can dispatch queries to other servers if the first server cannot 

15 provide the requested data. Caching of frequently-requested content can be provided at 
various locations through the network. Additional details on such network configurations 
can be found in application 09/343,104. 

As is familiar to those skilled in the arts, the foregoing methods may be 
performed using dedicated hardware at devices 12, 14 and 20, and/or through use of 

20 processors programmed in accordance with firmware or software, etc. In the latter case 
the processors may each include a CPU and associated memory, together with 
appropriate input and output devices/facilities. The software can be resident on a 
physical storage media such as disks, and can be loaded into the processors' memory for 
execution. The software includes instructions causing the CPU to perform the various 

25 processes detailed above. 

To provide a comprehensive disclosure without unduly lengthening this 
specification, applicant incorporates by reference the patents and applications cited 
above. 
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In view of the wide variety of embodiments to which the principles of our 
invention can be applied, it should be recognized that the detailed embodiments are 
illustrative only and should not be taken as limiting the scope of the invention. Rather, we 
claim as our invention all such embodiments as may come within the scope and spirit of 
the following claims, and equivalents thereto. 



